Abstract
Background and Aim: Machine learning enabled clinical decision support systems offer the potential to enhance the efficiency of decision-making processes in breast cancer multidisciplinary team meetings. We examine the circumstances where a traditional rule-based expert system may have advantages over machine learning enabled clinical decision support systems.
Methods: We compared the concordance of an expert system (Deontics) and a machine learning enabled clinical decision support system (Watson for Oncology) with the treatment recommendations of a gold standard consensus panel of breast surgeons and medical oncologists for 208 non-metastatic breast cancer patients, and for 165 patients deemed eligible for triage to an agreed standard of care, and ‘not for discussion at multidisciplinary team meeting’.
Results: The overall concordance between the Deontics clinical decision support system treatment plan recommendations and the gold standard consensus panel was 98% compared to 92% for the machine learning enabled clinical decision support system. Using a clinical decision tree, 79% of patients were eligible for triage to a standard of care and ‘not for discussion at multidisciplinary team meetings’; for these patients the concordance between the Deontics clinical decision support system and gold standard consensus panel was 98.8% (95% CI: 95.6-99.8%), whilst for the machine learning enabled clinical decision support system concordance was 78.8% (95% CI: 71.7-84.8%).
Conclusion: The high level of agreement between the Deontics clinical decision support system and clinical consensus suggests it may be acceptable for use in the breast multidisciplinary team pathway, whereas the level of disagreement observed for the machine learning enabled clinical decision support system would result in a clinically unsafe error rate if used to triage patients away from the multidisciplinary team meetings. These findings, if replicated in prospective studies in a routine clinical setting, could improve the efficiency of UK breast cancer diagnostic and treatment pathways.