Assessing the communication gap between AI models and healthcare professionals: explainability, utility and trust in AI-driven clinical decision-making. (arXiv:2204.05030v2 [cs.AI] UPDATED)

This paper contributes with a pragmatic evaluation framework for explainable
Machine Learning (ML) models for clinical decision support. The study revealed
a more nuanced role for ML explanation models, when these are pragmatically
embedded in the clinical context. Despite the general positive attitude of
healthcare professionals (HCPs) towards explanations as a safety and trust
mechanism, for a significant set of participants there were negative effects
associated with confirmation bias, accentuating model over-reliance and
increased effort to interact with the model. …

ai ai models arxiv communication decision explainability gap healthcare making trust