CORE-BEHRT: A Carefully Optimized and Rigorously Evaluated BEHRT

arXiv:2404.15201v1 Announce Type: new
Abstract: BERT-based models for Electronic Health Records (EHR) have surged in popularity following the release of BEHRT and Med-BERT. Subsequent models have largely built on these foundations despite the fundamental design choices of these pioneering models remaining underexplored. To address this issue, we introduce CORE-BEHRT, a Carefully Optimized and Rigorously Evaluated BEHRT. Through incremental optimization, we isolate the sources of improvement for key design choices, giving us insights into the effect of data representation and individual …

abstract arxiv bert core cs.lg design ehr electronic electronic health records fundamental health issue records release type