Cousins Of The Vendi Score: A Family Of Similarity-Based Diversity Metrics For Science And Machine Learning

arXiv:2310.12952v3 Announce Type: replace
Abstract: Measuring diversity accurately is important for many scientific fields, including machine learning (ML), ecology, and chemistry. The Vendi Score was introduced as a generic similarity-based diversity metric that extends the Hill number of order q=1 by leveraging ideas from quantum statistical mechanics. Contrary to many diversity metrics in ecology, the Vendi Score accounts for similarity and does not require knowledge of the prevalence of the categories in the collection to be evaluated for diversity. However, …

abstract arxiv chemistry cs.lg diversity ecology family fields hill ideas machine machine learning measuring metrics physics.chem-ph q-bio.pe science scientific type