Holistic Evaluation Metrics: Use Case Sensitive Evaluation Metrics for Federated Learning

arXiv:2405.02360v1 Announce Type: new
Abstract: A large number of federated learning (FL) algorithms have been proposed for different applications and from varying perspectives. However, the evaluation of such approaches often relies on a single metric (e.g., accuracy). Such a practice fails to account for the unique demands and diverse requirements of different use cases. Thus, how to comprehensively evaluate an FL algorithm and determine the most suitable candidate for a designated use case remains an open question. To mitigate this …

abstract accuracy algorithms applications arxiv case cs.dc cs.lg diverse evaluation evaluation metrics federated learning however metrics perspectives practice type unique