AB/BA analysis: A framework for estimating keyword spotting recall improvement while maintaining audio privacy. (arXiv:2204.08474v1 [cs.SD])

Evaluation of keyword spotting (KWS) systems that detect keywords in speech
is a challenging task under realistic privacy constraints. The KWS is designed
to only collect data when the keyword is present, limiting the availability of
hard samples that may contain false negatives, and preventing direct estimation
of model recall from production data. Alternatively, complementary data
collected from other sources may not be fully representative of the real
application. In this work, we propose an evaluation technique which we call …

ab analysis arxiv audio framework improvement privacy recall