Diagnosing Transformers: Illuminating Feature Spaces for Clinical Decision-Making

arXiv:2305.17588v3 Announce Type: replace-cross
Abstract: Pre-trained transformers are often fine-tuned to aid clinical decision-making using limited clinical notes. Model interpretability is crucial, especially in high-stakes domains like medicine, to establish trust and ensure safety, which requires human engagement. We introduce SUFO, a systematic framework that enhances interpretability of fine-tuned transformer feature spaces. SUFO utilizes a range of analytic and visualization techniques, including Supervised probing, Unsupervised similarity analysis, Feature dynamics, and Outlier analysis to address key questions about model trust and …

abstract arxiv clinical cs.ai cs.cl cs.lg decision domains engagement feature framework human interpretability making medicine model interpretability notes safety spaces transformer transformers trust type