Evaluating ethical and social risks from large models

Abstract:
Large generative models create the potential for many beneficial use cases, but they also raise significant ethical and social risks of harm. How can we know whether a large model is aligned with our expectations? In this talk, we first present a taxonomy of ethical and social risks developed by a multidisciplinary group of DeepMind researchers. We then turn to evaluation approaches to measure these risks. Surveying existing evaluation approaches - such as automated benchmarking and human adversarial testing, …

abstract benchmarking cases deepmind evaluation frameworks generative generative models highlight human large models researchers risks social talk taxonomy testing use cases