How to Evaluate Behavioral Models

arXiv:2306.04778v2 Announce Type: replace
Abstract: Researchers building behavioral models, such as behavioral game theorists, use experimental data to evaluate predictive models of human behavior. However, there is little agreement about which loss function should be used in evaluations, with error rate, negative log-likelihood, cross-entropy, Brier score, and squared L2 error all being common choices. We attempt to offer a principled answer to the question of which loss functions should be used for this task, formalizing axioms that we argue loss …

abstract agreement arxiv behavior building cross-entropy cs.gt cs.lg data entropy error experimental function game human likelihood loss negative predictive predictive models rate researchers type